On-line Step Size Adaptation Category: Algorithms and Architectures Sub-category: Online Learning Algorithms
نویسنده
چکیده
Gradient-based methods are often used for optimization. They form the basis of several neural network training algorithms, including backpropagation. They are known to be slow, however. Several techniques exist for the acceleration of gradient-based optimization, but very few of them are applicable to stochastic or real-time optimization. This paper proposes a new step size adaptation technique, designed speciically for accelerating stochastic gradient optimization and therefore also the real-time training of neural networks. The theoretical basis of the technique is discussed, and an experimental evaluation of the technique's performance is reported.
منابع مشابه
Assessment of the Performance of Clustering Algorithms in the Extraction of Similar Trajectories
In recent years, the tremendous and increasing growth of spatial trajectory data and the necessity of processing and extraction of useful information and meaningful patterns have led to the fact that many researchers have been attracted to the field of spatio-temporal trajectory clustering. The process and analysis of these trajectories have resulted in the extraction of useful information whic...
متن کاملConstructive Neural Network Learning Algorithms for Multi-category Pattern Classiication Constructive Neural Network Learning Algorithms for Multi-category Pattern Classiication
Constructive learning algorithms ooer an approach for incremental construction of potentially near-minimal neural network architectures for pattern classiication tasks. Such algorithms help overcome the need for ad-hoc and often inappropriate choice of network topology in the use of algorithms that search for a suitable weight setting in an otherwise a-priori xed network architecture. Several s...
متن کاملDetermination of Optimal Parameters for Finite Plates with a Quasi-Square Hole
This paper aims at optimizing the parameters involved in stress analysis of perforated plates, in order to achieve the least amount of stress around the square-shaped holes located in a finite isotropic plate using metaheuristic optimization algorithms. Metaheuristics may be classified into three main classes: evolutionary, physics-based, and swarm intelligence algorithms. This research uses Ge...
متن کاملGenetically Engineered ART Architectures
This chapter focuses on the evolution of ARTMAP architectures, using genetic algorithms, with the objective of improving generalization performance and alleviating the ART category proliferation problem. We refer to the resulting architectures as GFAM, GEAM, and GGAM. We demonstrate through extensive experimentation that evolved ARTMAP architectures exhibit good generalization and are of small ...
متن کاملOn - Line Learning for Active Pattern
| An adaptive on-line learning method is presented to faciliate pattern classiication using active sampling to identify optimal decision boundary for a stochas-tic oracle with minimum number of training samples. The strategy of sampling at the current estimate of the decision boundary is shown to be optimal compared to random sampling in the sense that the probability of convergence toward the ...
متن کامل